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ABSTRACT 



In 1996, a California high school district and two colleges 



assembled intersegmental teams to establish common assessment of 
college-readiness in mathematics and English. Subject area teams reviewed 
standards, rubrics, and current tests. A Joint Assessment Team ( JAT) explored 
the Golden State Exam (GSE) in Written Composition as a common tool . Its 
format and scoring were similar to college assessments, and it was available 
statewide. In 1997, 250 high school students took the GSE. A holistic rubric 
was developed. Essays were read and scored, and data sets were matched. There 
was a strong correlation, which convinced the team to continue with the GSE. 
In 1998, 300 more essays were read and scored holistically and 
diagnostically. Holistic grades were correlated with GSE scores, yielding a 
strong correlation. In 1999, 3,300 high school juniors took the GSE. The 
correlation was still strong. Letters were sent to students with diagnostic 
feedback and GSE scores, noting the relevance for college placement. JAT 
implementation has had several positive effects in the school district. While 
the GSE has changed from year to year, it has remained a reliable instrument. 
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The Problem 

For years the California educational system has experienced a significant disparity 
between the skills higher education required of entering freshmen and the skills many of 
those students brought with them. Annual reports decried the percentage of new students 
sentenced to remedial classes in English and mathematics. The California State University 
even adopted a plan to reduce remedial students from fifty percent to ten percent by the year 
2007. 

Unfortunately, the data provided to local high schools came years after the students 
graduated and provided no diagnostic feedback. Consequently, the information was neither 
timely nor efficacious. 

Program Background 

In the fall of 1 996, the Kern High School District, Bakersfield College, and California 
State University, Bakersfield, assembled inter-segmental teams to establish common 
assessment of college-readiness in mathematics and English language. Subject area teams 
met to review standards, rubrics, and current tests. The English program is the subject of this 
paper. 

The Setting 

The program’s geography is currently defined by the boundaries of the Kern High 
School District. While expansion into other Kern County districts and CSUB/BC service 
areas is desired, it is not yet an accomplished fact. 

The Kern High School District includes fourteen comprehensive high schools and 
many specialized schools. The population of the comprehensive schools ranges from less 
than 700 students to more than 2,750. The total 9-12 population of KHSD exceeds 28,000. 
As such, it is the primary feeder to both Bakersfield College and California State University, 
Bakersfield. 

This large student population combined with relative geographic isolation has created 
an ideal situation for inter- segmental efforts. Indeed, there has been a long history of 
articulation among the three institutions, though usually in bilateral fashion. Since 1990, BC 
and CSUB have articulated a great deal of their respective curricula. While several faculty 
members work at both colleges, more systematic efforts were established to maintain 
common standards. Beyond articulation meetings, the schools shared common scoring 
sessions. Ultimately, the two schools decided that scores on placement tests would be 
mutually honored. 

The Joint Assessment Team 

The formation of the Joint Assessment Team in 1 996 built on the work of BC and 
CSUB in defining and identifying quality writing. A team of English instructors from the 
three institutions met to review data and instruments currently available. While all three 
administrations actively supported the team, the instructors were the sole decision-makers. 

After several months of collaborative study, the committee decided to explore the 
Golden State Exam in Written Composition as a common tool. The GSE was attractive 
because its format and scoring were similar to the college assessments, and it was available 
statewide. Should research indicate its utility, students from beyond the local area could also 
benefit. 



Program History 

Having settled on the GSE as the first choice, it was critical to explore the program’s 
scoring patterns with those of the college and university. In the spring of 1997, more than 
250 KHSD students took the exam. Copies of the responses were secured for future scoring. 
Meanwhile, the team developed a holistic rubric (Appendix A), which met the approval of all 
three agencies. This rubric used a six-point scale parallel to that used by the GSE. During 
the summer of 1997, the essays were read and scores recorded. When the state GSE scores 
arrived, the data sets were matched up and a correlation of .737 was calculated. 

This strong correlation convinced the team to continue down the GSE path. Prior to 
GSE testing in 1998, work was completed on a diagnostic scoring model (Appendix B). In 
1998, another 300 essays were read and scored both holistically and diagnostically. The 
holistic grades were again correlated with GSE scores yielding a coefficient of .723. Letters 
were sent to all participating students who were seniors the following fall. These letters 
indicated the areas of potential improvement in the students’ writing. 

In January of 1999, more than 3300 KHSD juniors took the GSE. Juniors were 
targeted in order to return feedback to both students and teachers early in the senior year. 
Only the long essay was considered for further study. A sample of 1221 essays was scored 
holistically while all were scored diagnostically. The resulting correlation of .641 was lower 
than in previous years, though still strong. GSE scores were again slightly lower than local 
JAT scores, a good thing for the continued support of collegiate faculty. 

The fall 1999 letters to students included not only the diagnostic feedback, but the 
actual GSE score and an interpretation of its relevance to placement at BC or CSUB 
(Appendix C). Additionally, a duplicate copy of each student’s report was provided to the 
senior English teacher along with copies of the scoring rubrics and an explanation of the 
Diagnostic Scoring Guide (Appendix D.) 

The weakening of the correlation over the three years of the study may be due to 
several factors. Each scoring session was marked by somewhat different tests and slightly 
different scoring procedures. In 1997, the GSE consisted of two long essays. Both were read 
by JAT readers. Two readers scored the paper; non-adjacent scores were taken to third 
readers and adjacent scores were averaged. In 1998, the test changed to a long essay, a short 
essay, and a multiple-choice element. While the GSE score was mapped from all three 
elements, JAT only read the essays. That year, no score discrepancies were allowed, and all 
such papers were taken to multiple readers. In 1999, the JAT teams read only the long essay, 
roughly half the test. Adjacent scores were allowed to stand; third readings were only used 
to validate scoring patterns and resolve non-adjacent discrepancies. 

In the fall of 1999, an additional element was added to the JAT process when 1 1 th 
grade English teachers were invited to a sample scoring session. “Live” essay samples were 
taken from current students, and the JAT team supervised a day-long scoring. The day 
concluded with a critique of the process and suggestions for refining the rubrics and 
explanations. What had started as an “early-warning” test had become an inter-segmental in- 
service. 
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View from the High School 

The implementation of JAT has had several positive effects in the KHSD. The 
existence of a commonly measured standard for college-bound writing has proven beneficial 
to administration, faculty, and students. Beyond the early holistic feedback, however, the 
diagnostic scoring has proven to be the salient benefit. 

One of the main reasons for developing and implementing the entire JAT plan was to 
report the results of the exam to the high school students who wrote the exam. Without the 
information that the Diagnostic Scoring Guide afforded the students, the information reported 
by the Holistic Scoring Guide was just another score. When students received a GSE 
Certificate of Recognition, Honor, or High Honor, they also received a GSE/JAT Report that 
informed them of the areas upon which they could improve their writing according to the 
Diagnostic Scoring Guide. The real learning takes place when students sit down with 
teachers and discuss the points outlined on the GSE/JAT Report in conjunction with essay 
writing in general. When students are able to connect essay writing with the Diagnostic 
Scoring Guide, it becomes a powerful teaching tool. 

The Diagnostic Scoring Guide has also become a powerful learning tool for the 
teachers themselves. Throughout the process of developing the guide, input from teachers at 
many school sites had to be considered. This was an impetus to get teachers thinking about 
diagnostic issues in conjunction with essay rating, grading, and assessment in general. Once 
a draft of the guide was completed, it was tested by the JAT committee before it was 
introduced to a larger group of teachers at an in-service. Thereafter, it was refined and 
drafted into a working assessment tool before being used as an actual diagnostic tool. 
Throughout this entire process, university, college, and high school instructors met, worked 
together, and learned from each other in order to produce the best guide possible. 

The summer scoring sessions put the Diagnostic Scoring Guide to the test as both an 
assessment tool and a teaching/leaming tool. English instructors from the three institutions 
(CSUB, BC, KHSD) came together to grade the selected GSE exams, and the instructors 
were able to communicate with one another about the writing process and the quality of the 
product it should yield. By the end of the sessions, the instructors all had a much better idea 
of how to use the Diagnostic Scoring Guide as an assessment tool and as a leaming/teaching 
tool. At the end of the sessions, discussion is always held to determine how the guides can 
be made even better. 

The Diagnostic Scoring Guide was first developed by the JAT committee as 
an assessment and feedback tool for the GSE essay exam. It has turned into much more than 
an assessment tool. The guide has now become an assessment/teaching/leaming tool that 
helps both instructors and students. Students are becoming more aware of how they can 
become better writers. Instructors are discovering common ground to emphasize in the 
instruction of writing. The Diagnostic Scoring Guide has become a valuable tool for all 
involved in the writing process at the high school level. 

View from the College and University 

The most direct benefit to higher education is the luxury afforded by many students 
arriving pre-assessed, but the real import lies elsewhere. True, students who have mastered 
college-level writing skills need only test once to earn GSE recognition and to be placed in 
English courses at BC (see Appendix E) or CSUB, but those who do not score well are 
perhaps even better served. Students who have not mastered these skills receive diagnostic 



feedback to help them better prepare for the standard placement test in their senior year. 
Successful testers will have a reduced testing load while unsuccessful students are given an 
opportunity to see specific feedback on their strengths and weaknesses in writing. So, this 
testing service serves all students-both those who are well prepared for college and those 
who are not. 

While the GSE has changed from year to year, it has been found to be a reliable 
instrument. The holistic scores recorded by the state readers are a close match to the 
assessment standards at CSUB and BC and the scores awarded by our team of high school 
and college readers. When there is a difference, the state scores tend to be a bit more 
rigorous, but the students can always retest using the standard assessment test at either 
college if they are not satisfied with the GSE score. While the statistical data are 
encouraging, some English types are more comfortable with anecdotal data. In the initial 
year of the project, the BC representative looked up individual students who had taken both 
GSE and the BC writing exam. A remarkable consistency was noted, and the GSE score had 
placed students at an appropriate and successful level. 

Beyond the obvious benefit to students, this assessment project furnishes the local 9- 
12 faculty various opportunities in our grading sessions to discuss standards and strategies 
for reaching the standards. No longer are the college standards a mystery to be solved, but a 
goal for us all to work toward. This project will ultimately raise the literacy level in the 
community and the inter-segmental benefits far outweigh any drawbacks encountered. 

Future Program Design 

With GSE becoming an anchor in the junior curriculum, the intent is to work the 
process down the grade levels. Reading GSE papers will continue only on a sampling basis 
to maintain the link to the college placement tests. The broader diagnostic reading will be 
moved to a local GSE-like sample taken at the end of the sophomore year with scoring 
throughout the summer. Juniors will return to school and receive specific recommendations 
to help them prepare for the GSE in January. Their January motivation will be enhanced by 
the tangible benefit of direct college placement should their score suffice. 

As more students become involved in the GSE/JAT process, schools are better able to 
use these data to help guide students. One KHSD school doubled its enrollment in AP 
English after noting how many students were already writing at a level deemed strong by the 
university. The same school also increased enrollments in its senior composition course due 
to student demand to improve writing skills. Such indicators give hope that the JAT process 
will effect significant improvement in the writing skills of future Kern High School District 
graduates. 



APPENDIX A 



Holistic Scoring Guide 

Circle your holistic score - and use the diagnostic scoring guide to identify problems in the writing sample. 

Score of 6: Superior Essay 

Responsive to writing task 
Thoughtful and interesting development 
Coherently organized 
Effective, mature style 

Relatively free from errors in sentence structure, usage, and mechanics 



Score of 5: Strong Essay 

Relatively responsive to writing task 
Fully developed 
Well organized 
Readable style 

Relatively free from errors in sentence structure, usage, and mechanics 



Score of 4: Adequate Essay 

Generally responsive to writing task 
Adequately developed 
Clearly organized 
Some sense of style 

Some errors in sentence structure, usage, and/or mechanics 



Score of 3: Mediocre Essay (has one of more of the following problems) 

Limited understanding of the writing task 

Inadequate development 

Unclear organization 

Limited control of syntax and language 

Numerous errors in sentence structure, usage, and mechanics 



Score of 2: Weak Essay (has one or more of the following problems) 

Inadequate understanding of the topic and writing task 

Underdeveloped 

Weak organization 

Inadequate control of syntax and language 

Numerous and varied errors in sentence structure, usage, and mechanics 

Score of 1: Incompetent Essay (has one or more of the following problems) 

Misunderstanding of topic and writing task 

Little or no development 

Disorganized and incoherent 

Little or no control of syntax and language 

Serious and varied errors in sentence structure, usage, and mechanics 



APPENDIX B 



Diagnostic Scoring Guide 

In this guide you will find numbers to correspond with common problems found in student 
writing. In addition to giving the essay a holistic score (see the Holistic Scoring Guide), use 
these numbers to identify areas for improvement. On the following pages are examples and 
explanations corresponding to each of the numbered categories. 

Write these numbers under the circled holistic score. 

Response to the Question 

1. understanding the topic 

2. awareness of audience 

Development 

3. support (evidence) 

4. focus 

5. repetition 

Organization 

6. thesis 

7. coherence/logic 

8. expository essay form 

Style 

9. sentence variety 

10. transitions 

11. vocabulary 

12. appropriate language (level of diction, slang, cliches, jargon, idioms) 

13. wordiness 

Correctness 

14. spelling 

15. confused words/modifier forms 

16. punctuation 

17. sentence structure/syntax 

18. verb forms (including subject/verb agreement, tense) 

19. noun/adjective forms 

20. pronoun usage 
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APPENDIX C 



Student Letter 



Date: 


October 15, 1999 


To: 






Stockdale High School 


From: 


Paul Hoagland, Director 




Research & Planning 



Last year you took the Golden State Exam in Written Expression. This test can award state 
honors, but is also honored as the English writing placement test at both BC and CSUB. 
Papers were scored 1-6. Scores of 5 and 6 indicate full readiness for college-level writing. 
Scores of 1-4 indicate ranges of improvement needed this important senior year. Your 
score was 2. 



As a further service to you, your essay was read by a joint team of college and high school 
instructors to identify areas in which you can strengthen your writing in preparation for 
college. Papers were evaluated for effectiveness in the following areas: 



Response to the Question: 
Development: 
Organization: 
Style: 



Correctness: 



understanding the topic, awareness of audience 
support, focus, repetition 
theses, coherence/logic, expository essay form 
sentence variety, transitions, vocabulary, appropriate 
language (level of diction, slang, cliches, jargon, 
idioms), wordiness 

spelling, confused words/modifier forms, punctuation, 
sentence structure/syntax, verb forms and pronoun 
usage 



According to the evaluation you can strengthen your writing in the following areas: 

spelling 

confused words 
punctuation 
sentence structure 
verb forms 



We hope you will find this information helpful in working towards your educational goals. 
Your English teacher is an invaluable resource in strengthening your writing. 



APPENDIX D 



Diagnostic Scoring Guide - Examples/explanations 

1. understanding the topic 

The essay’s thesis and development show that the student understands the essay 
prompt. 

2. awareness of audience 

The essay is directed toward its intended audience, in this case evaluators from high 
schools, colleges, and universities. While the essay needn’t be formal, getting too 
personal, chatty, or colloquial is not recommended. 

3. support (evidence) 

The essay should have enough relevant and specific supporting evidence to develop 
its paragraphs. Generalities and vague details do not constitute good support. 

4. focus 

The supporting material in the paragraphs of the essay should relate specifically to the 
thesis statement, and the writer should maintain a balance between specific detail and 
analysis. 

5. repetition 

When a sentence or paragraph contains the same words and ideas more than once, the 
essay’s development may suffer although repetition may also be used for stylistic 
effect. 

6. thesis 

The essay must have a clearly written thesis statement. Typically this statement 
appears in the introductory paragraph; however, an accomplished writer may succeed 
in delaying the thesis or using an implied thesis. 

7. coherence/logic 

The thesis statement and supporting evidence should follow logically and form a 
coherent argument. 

8. expository essay form 

The essay should contain a clearly written thesis statement and be divided into 
adequately developed paragraphs with an identifiable topic. Overall, the essay should 
have a discernible plan. 

9. sentence variety 

The essay should contain a variety of simple, compound, complex, and compound- 
complex sentences. The writer should also vary the beginnings of his/her sentences. 
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10. transitions 

The writer should use transitional sentences and devices between ideas and 
paragraphs to allow them to flow into one another smoothly. However, the writer 
should avoid the use of repetitive and formulaic transitional devices. 



11. vocabulary 

Keeping the audience in mind, the writer should use vocabulary appropriate to the 
level and purpose of the writing and include words other than those used in common 
colloquial situations. 

12. appropriate language (level of diction, slang, cliches, jargon, idioms) 

Writers should use standard diction, including appropriate prepositions and articles. 
The essay should only use slang, jargon, and cliches for effect. In this case, the writer 
should put these expressions in quotation marks to acknowledge their use. 
Furthermore, writers should avoid using vague words such as “thing” and 
“everything.” 

13. wordiness 

Economy within sentences is usually most effective. “Padding” of any sort makes 
prose wordy and difficult to follow. 



14. spelling 

This category includes all misspellings. 



15. confused words/modifier forms 

This category should be designated if the writer confuses words such as affect/effect, 
too/to/two, its/it’s, their/there/they’re, less/fewer, etc. 



16. punctuation 

Writers should follow the basic conventions of English punctuation. Specifically, a 
grasp of standard comma rules should be clear in the essay: 

If they study, they will do well. (dep. clause, ind. clause) 

They write well, and they read well. (ind. clause, coord, conj. ind. clause) 
Reading, writing, and speaking . . . (items in a series) 

Bakersfield, California (city, state) 

The semicolon, if used, should separate independent clauses: 

They study hard; they do well. (ind. clause ; ind. clause) 

Apostrophes in possessive forms and in contractions should be used correctly: 
the student ’s essay/the man ’s hat (singular possessive) 
the students ’ essays/the people ’s choice (plural possessive) 
don ’t/can ’t (contractions) 

Apostrophes should not be used to form plurals or the singular verb form. 

Quotation marks should be used correctly: 

“Who are they?” 

He put real emphasis on the word “they. ” 

“I write quite well, ” the student boasted. 
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End punctuation should be used appropriately. 

Doubling and tripling end punctuation marks is not necessary. 



17. sentence structure/syntax 

Writers should avoid fragments and run-ons and use parallel structure correctly. 
These examples show typical errors: 

Because students must be able to write well. (frag. - dep. clause) 

The young lady wearing the pretty dress. (frag. - noun phrase) 

We write essays we read literature. (fused sent. - needs , conjunction) 

We write essays, we read literature. (comma splice - needs , conjunction) 

She loves to read, to write, and speaking. (faulty parallelism — to speak — 

not speaking) 



18. verb forms (subject/verb agreement, tense) 

Writers should make sure that their verbs agree with their subjects and that their verb 
forms and tenses are correct and consistent. 

These examples show typical errors: 

The student with her friends watch a movie. 

Jobs is what teenagers need. 

A good writer does not shift verb tense: 

I was reading, and he plays his stereo. 

Correct verb forms must be used: 

They should of seen the movie. 

We seen the movie yesterday. 



{watches - student is the subject) 
{are - jobs is the subject) 

{played- remain in past tense) 

(should have — “of’ is not a verb) 
(saw or have seen) 



19. noun/adjective forms 

Writers should make sure that their noun and adjective forms are correct. 

Plural noun forms: man/men, woman/women 

Adjective forms: good, better, best; quick, quicker, quickest 

Writers should not confuse adjective and adverb forms: 

bad, badly; good, well; many, much 

Noun/adjective combinations should be done correctly. 

our lives, not our life 



20. pronoun usage 

Pronoun shifts (from third to second person, or from singular to plural, for example) 
should be avoided. 

Vague pronoun use, such as the use of “it” and “that” and other pronouns without 
clear 

antecedents, should be avoided. 

Overuse and vague use of “you” should be avoided. 

These examples show some typical errors: 

Pronouns should agree with their antecedents: 

If a person writes well, they will succeed. {he or she - not they) 

Reflexive and intensive pronouns should be used correctly: 

The group consisted of him, her, and myself. {me - not myself) 

I, myself, would not go with them. {myself - correct intensified 



I thought silently to myself. {myself- correct reflexive) 
Pronouns should be written in the correct case. 
who/ whom 

The drug dealers who the police arrest often do not serve time, (whom - correct case,) 



APPENDIX E 




Special Placement in English Composition 



Name 



SS# 



Current Placement Level 



Change level to 



ESL OYes DNo 



High School Graduate: □ Yes □ No Year of High School Graduation 



Reason for change to English 1 A (06): 

□ Counselor determination on the basis of College Board Advanced Placement Program (3,4,5 on Language and Composition or 
Composition and Literature exams). 

□ Score of 600 or above on the College Board SAT II or Achievement Test in English composition with essay. 

□ Score of 550 (score of 470 prior to 4/95) or above on verbal section of SAT/SAT 1 (plus a score of 10, 1 1 , or 12 on the English 

Placement Essay written at BC). 

□ Score of 22 or above on ACT English Usage Test (plus a score of 10, 1 1, or 12 on the English Placement Essay written at BC). 

□ CSUB English 110 placement on the ETP (total score of 155-180 for CSU System which must include essay). 

□ Score of 5 or 6 on the English component of the GSE exam (recorded on high school transcript). 

□ English 1 A placement on the UC Subject A Exam. 

□ Completion of an introductory writing course equivalent to English 1 at another accredited college (with a grade of C or better). 

□ Meets criteria indicated below under multiple measures. 

□ Other 



Reason for change to English 1 (05): 

□ CSUB English 100 placement on the EPT (total score of 147-154 which must include essay). 

□ Score of 4 on the English component of the GSE exam (recorded on the high school transcript). 

□ Completion of basic skills writing course equivalent to English 60 at another accredited college (with a grade of C or better). 

□ Meets criteria indicated below under multiple measures. 

□ Other 



Reason for change to English 60 (04): 

□ Completion of a developmental writing course equivalent to Academic Development 68 at another accredited college (with a grade 
of C or better). 

□ Meets criteria indicated below under multiple measures. 

□ Other 



Multiple Measures Criteria 

□ Has met high school cumulative grade point average of 3.0 or higher 
AND must meet two out of three of the following criteria... 

□ Three to four years of high school English. 

□ Education plans include transfer to a four-year college. 

□ Full-time enrollment (12 or more units per semester). 



Counselor Signature Date 



White-Counseling Yellow- English Department Pink-Student 
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